Deep convolutional neural network with self-transfer learning

ABSTRACT

Systems and techniques for facilitating a deep convolutional neural network with self-transfer learning are presented. In one example, a system includes a machine learning component, a medical imaging diagnosis component and a visualization component. The machine learning component generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data. The machine learning component also performs a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The medical imaging diagnosis component determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. The visualization component generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

TECHNICAL FIELD

This disclosure relates generally to artificial intelligence.

BACKGROUND

Artificial Intelligence (AI) can be employed for classification and/oranalysis of digital images. For instance, AI can be employed for imagerecognition. In certain technical applications, AI can be employed toenhance medical imaging diagnosis. Diseases of a patient can beclassified, for example, by analyzing medical images of the patientusing a deep neural network. In an example, region-of-interest baseddeep neural networks can be employed to localize a disease in ananatomical region of a patient. However, accuracy and/or efficiency of aclassification and/or an analysis of digital images using conventionalartificial techniques is generally difficult to achieve. Furthermore,conventional artificial techniques for classification and/or analysis ofdigital images generally requires labor-intensive processes such as, forexample, pixel annotations, voxel level annotations, etc. As such,conventional artificial techniques for classification and/or analysis ofdigital images can be improved.

SUMMARY

The following presents a simplified summary of the specification inorder to provide a basic understanding of some aspects of thespecification. This summary is not an extensive overview of thespecification. It is intended to neither identify key or criticalelements of the specification, nor delineate any scope of the particularimplementations of the specification or any scope of the claims. Itssole purpose is to present some concepts of the specification in asimplified form as a prelude to the more detailed description that ispresented later.

According to an embodiment, a system includes a machine learningcomponent, a medical imaging diagnosis component and a visualizationcomponent. The machine learning component generates learned medicalimaging output regarding an anatomical region based on a convolutionalneural network that receives medical imaging data. The machine learningcomponent also performs a plurality of sequential downsampling andupsampling of the medical imaging data associated with convolutionallayers of the convolutional neural network. The medical imagingdiagnosis component determines a classification and an associatedlocalization for a portion of the anatomical region based on the learnedmedical imaging output associated with the convolutional neural network.The visualization component generates a multi-dimensional visualizationassociated with the classification and the localization for the portionof the anatomical region.

According to another embodiment, a method is provided. The methodprovides for receiving, by a system comprising a processor, medicalimaging data for a patient body. The method also provides forperforming, by the system, iterative sequential downsampling andupsampling of the medical imaging data associated with convolutionallayers of a convolutional neural network to generate learned medicalimaging output regarding the patient body. Furthermore, the methodprovides for classifying, by the system, a disease for a portion of thepatient body based on the learned medical imaging output associated withthe convolutional neural network. The method also provides forgenerating, by the system, a multi-dimensional visualization associatedwith the classifying of the disease for the portion of the patient body.

According to yet another embodiment, a method is provided. The methodprovides for receiving, by a system comprising a processor, medicalimaging data that comprises a set of medical images. The method alsoprovides for training, by the system, a convolutional neural network byperforming iterative sequential downsampling and upsampling of themedical imaging data associated with convolutional layers of theconvolutional neural network. Furthermore, the method provides forgenerating, by the system, a set of filter values for the convolutionalneural network based on the iterative sequential downsampling andupsampling of the medical imaging data.

The following description and the annexed drawings set forth certainillustrative aspects of the specification. These aspects are indicative,however, of but a few of the various ways in which the principles of thespecification may be employed. Other advantages and novel features ofthe specification will become apparent from the following detaileddescription of the specification when considered in conjunction with thedrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Numerous aspects, implementations, objects and advantages of the presentinvention will be apparent upon consideration of the following detaileddescription, taken in conjunction with the accompanying drawings, inwhich like reference characters refer to like parts throughout, and inwhich:

FIGS. 1-6 illustrate a high-level block diagram of an example deeplearning component, in accordance with various aspects andimplementations described herein;

FIG. 7 illustrates an example system associated with a springself-transfer learning network, in accordance with various aspects andimplementations described herein;

FIG. 8 illustrates an example system associated with an inference phasefor a spring self-transfer learning network, in accordance with variousaspects and implementations described herein;

FIG. 9 illustrates an example system associated with sequentialupsampling and downsampling for a spring self-transfer learning network,in accordance with various aspects and implementations described herein;

FIG. 10 illustrates an example multi-dimensional visualization, inaccordance with various aspects and implementations described herein;

FIG. 11 illustrates an example user interface, in accordance withvarious aspects and implementations described herein;

FIG. 12 depicts a flow diagram of an example method for facilitating adeep convolutional neural network with self-transfer learning, inaccordance with various aspects and implementations described herein;

FIG. 13 depicts a flow diagram of an example method for facilitatingtraining of a deep convolutional neural network with self-transferlearning, in accordance with various aspects and implementationsdescribed herein;

FIG. 14 is a schematic block diagram illustrating a suitable operatingenvironment; and

FIG. 15 is a schematic block diagram of a sample-computing environment.

DETAILED DESCRIPTION

Various aspects of this disclosure are now described with reference tothe drawings, wherein like reference numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of one or more aspects. It should beunderstood, however, that certain aspects of this disclosure may bepracticed without these specific details, or with other methods,components, materials, etc. In other instances, well-known structuresand devices are shown in block diagram form to facilitate describing oneor more aspects.

Systems and techniques for performing self-transfer learning associatedwith a deep convolutional network are presented. For example, ascompared to conventional artificial intelligence (AI) techniques, thesubject innovations provide for a novel weakly supervised AI frameworkwith self-transfer learning. The novel weakly supervised AI frameworkcan perform machine learning (e.g., deep learning) associated with adeep convolutional network by employing image-level labels to classifyand/or localize a disease over a particular anatomical region associatedwith medical imaging data. In an aspect, the novel weakly supervised AIframework can comprise shared convolutional layers containing sequentialdown sampling and/or sequential up sampling. Additionally oralternatively, the novel weakly supervised AI framework can comprisefully connected layers and/or class activation maps. As such, byemploying the novel weakly supervised AI framework to analyze medicalimaging data, detection and/or localization of diseases for a patientassociated with the medical imaging data can be improved. Furthermore,accuracy and/or efficiency for classification and/or analysis of digitalimages (e.g., medical imaging data) can be improved. Moreover,effectiveness of a machine learning model for classification and/oranalysis of digital images (e.g., medical imaging data) can be improved,performance of one or more processors that execute a machine learningmodel for classification and/or analysis of digital images (e.g.,medical imaging data) can be improved, and/or efficiency of one or moreprocessors that execute a machine learning model for classificationand/or analysis of digital images (e.g., medical imaging data) can beimproved.

Referring initially to FIG. 1, there is illustrated an example system100 that provides a deep convolutional neural network with self-transferlearning, according to an aspect of the subject disclosure. The system100 can be employed by various systems, such as, but not limited tomedical device systems, medical imaging systems, medical diagnosticsystems, medical systems, medical modeling systems, enterprise imagingsolution systems, advanced diagnostic tool systems, simulation systems,image management platform systems, care delivery management systems,artificial intelligence systems, machine learning systems, neuralnetwork systems, modeling systems, aviation systems, power systems,distributed power systems, energy management systems, thermal managementsystems, transportation systems, oil and gas systems, mechanicalsystems, machine systems, device systems, cloud-based systems, heatingsystems, HVAC systems, medical systems, automobile systems, aircraftsystems, water craft systems, water filtration systems, cooling systems,pump systems, engine systems, prognostics systems, machine designsystems, and the like. In one example, the system 100 can be associatedwith a viewer system to facilitate visualization and/or interpretationof medical imaging data. Moreover, the system 100 and/or the componentsof the system 100 can be employed to use hardware and/or software tosolve problems that are highly technical in nature (e.g., related toprocessing digital data, related to processing medical imaging data,related to medical modeling, related to medical imaging, related toartificial intelligence, etc.), that are not abstract and that cannot beperformed as a set of mental acts by a human.

The system 100 can include a deep learning component 102 that caninclude a machine learning component 104, a medical imaging diagnosiscomponent 106 and a visualization component 108. Aspects of the systems,apparatuses or processes explained in this disclosure can constitutemachine-executable component(s) embodied within machine(s), e.g.,embodied in one or more computer readable mediums (or media) associatedwith one or more machines. Such component(s), when executed by the oneor more machines, e.g., computer(s), computing device(s), virtualmachine(s), etc. can cause the machine(s) to perform the operationsdescribed. The system 100 (e.g., the deep learning component 102) caninclude memory 112 for storing computer executable components andinstructions. The system 100 (e.g., the deep learning component 102) canfurther include a processor 110 to facilitate operation of theinstructions (e.g., computer executable components and instructions) bythe system 100 (e.g., the deep learning component 102).

The deep learning component 102 (e.g., the machine learning component104) can receive medical imaging data (e.g., MEDICAL IMAGING DATA shownin FIG. 1). The medical imaging data can be two-dimensional medicalimaging data and/or three-dimensional medical imaging data generated byone or more medical imaging devices. For instance, the medical imagingdata can be electromagnetic radiation imagery captured via a set ofsensors (e.g., a set of sensors associated with a medical imagingdevice). In certain embodiments, the medical imaging data can be aseries of electromagnetic radiation imagery captured via a set ofsensors (e.g., a set of sensors associated with a medical imagingdevice) during an interval of time. The medical imaging data can bereceived directly from one or more medical imaging devices.Alternatively, the medical imaging data can be stored in one or moredatabases that receives and/or stores the medical imaging dataassociated with the one or more medical imaging devices. A medicalimaging device can be, for example, an x-ray device, a computedtomography (CT) device, another type of medical imaging device, etc. Themachine learning component 104 can perform a machine learning process(e.g., an artificial intelligence process for machine learning) based onthe medical imaging data. In an aspect, the machine learning component104 can perform deep learning to facilitate classification and/orlocalization of one or more diseases associated with the medical imagingdata. In another aspect, the machine learning component 104 can performdeep learning based on a convolutional neural network that receives themedical imaging data.

In an embodiment, the machine learning component 104 can perform atraining phase for the machine learning process. For example, themedical imaging data can be a set of medical images (e.g., a set ofx-ray images, etc.) stored in a data store. Furthermore, the machinelearning component 104 can perform the training phase for the machinelearning process based on the set of medical images stored in a datastore to train a neural network model (e.g., a neural network model forthe convolutional neural network). In certain embodiments, the machinelearning component 104 can employ a first portion of the medical imagingdata for training associated with the convolutional neural network, asecond portion of the medical imaging data for validation associatedwith the convolutional neural network, and a third portion of themedical imaging data for testing associated with the convolutionalneural network. Additionally or alternatively, the machine learningcomponent 104 can randomly select a set of medical images from atraining set associated with the medical imaging data for dataaugmentation associated with the medical imaging data. In an aspect, themachine learning component 104 can modify an orientation of the set ofmedical images for the data augmentation associated with the medicalimaging data. In one example, the machine learning component 104 canmodify the set of medical images through at least one affinetransformation for the data augmentation associated with the medicalimaging data. In another embodiment, the machine learning component 104can perform an inference phase. For example, the medical imaging datacan be a medical image for an anatomical region of a patient associatedwith the medical image. Furthermore, the machine learning component 104can perform the training phase for the machine learning process based onthe medical image. For an inference phase associated with the machinelearning component 104, the machine learning component 104 can generatelearned medical imaging output regarding an anatomical region based onthe convolutional neural network that receives medical imaging data.

In an aspect, the machine learning component 104 can employ a springnetwork of convolutional layers. The machine learning component 104 canemploy the spring network of convolutional layers to generate thelearned medical imaging output based on the medical imaging data. In anaspect, the machine learning component 104 can generate the learnedmedical imaging output based on a first convolutional layer processassociated with sequential downsampling of the medical imaging data anda second convolutional layer process associated with sequentialupsampling of the medical imaging data. The spring network ofconvolutional layers can include the first convolutional layer processassociated with the sequential downsampling and the second convolutionallayer process associated with sequential upsampling. In one example, themachine learning component 104 can perform a plurality of sequentialdownsampling and upsampling of the medical imaging data associated withconvolutional layers of the convolutional neural network. The springnetwork of convolutional layers employed by the machine learningcomponent 104 can alter convolutional layer filters similar tofunctionality of a spring. For instance, the machine learning component104 can analyze the medical imaging data based on a first convolutionallayer filter that comprises a first size, a second convolutional layerfilter that comprises a second size that is different than the firstsize, and a third convolutional layer filter that comprises the firstsize associated with the first convolutional layer filter.

In certain embodiments, the machine learning component 104 can extractinformation that is indicative of correlations, inferences and/orexpressions from the medical imaging data based on the spring network ofconvolutional layers. The machine learning component 104 can generatethe learned medical imaging output based on the execution of at leastone machine learning model associated with the spring network ofconvolutional layers. The learned medical imaging output generated bythe machine learning component 104 can include, for example, learning,correlations, inferences and/or expressions associated with the medicalimaging data. In an aspect, the machine learning component 104 canperform learning with respect to the medical imaging data explicitly orimplicitly using the spring network of convolutional layers. The machinelearning component 104 can also employ an automatic classificationsystem and/or an automatic classification process to facilitate analysisof the medical imaging data. For example, the machine learning component104 can employ a probabilistic and/or statistical-based analysis (e.g.,factoring into the analysis utilities and costs) to learn and/orgenerate inferences with respect to the medical imaging data. Themachine learning component 104 can employ, for example, a support vectormachine (SVM) classifier to learn and/or generate inferences for medicalimaging data. Additionally or alternatively, the machine learningcomponent 104 can employ other classification techniques associated withBayesian networks, decision trees and/or probabilistic classificationmodels. Classifiers employed by the machine learning component 104 canbe explicitly trained (e.g., via a generic training data) as well asimplicitly trained (e.g., via receiving extrinsic information). Forexample, with respect to SVM's, SVM's can be configured via a learningor training phase within a classifier constructor and feature selectionmodule. A classifier can be a function that maps an input attributevector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongsto a class —that is, f(x)=confidence(class).

To facilitate localization of one or more diseases associated with themedical imaging data, the machine learning component 104 can perform alocal pooling process for an activation map associated with aconvolutional layer of the convolutional neural network prior toperforming a global pooling process associated with the convolutionalneural network. Additionally or alternatively, the machine learningcomponent 104 can generate the learned medical imaging output based on aclass activation mapping process that applies a set of weights to a setof heat maps associated with the medical imaging data. Additionally oralternatively, the machine learning component 104 can process themedical imaging data based on one or more regularization techniques toclassify one or more portions of the medical imaging data. In an aspect,the machine learning component 104 can also merge a set of classifierlayers associated with the convolutional neural network and a set ofactivation maps associated with the convolutional neural network togenerate the learned medical imaging output.

The medical imaging diagnosis component 106 can employ informationprovided by the machine learning component 104 (e.g., the learnedmedical imaging output) to classify and/or localize a disease associatedwith the medical imaging data. In an embodiment, the medical imagingdiagnosis component 106 can determine a classification and an associatedlocalization for a portion of the anatomical region based on the learnedmedical imaging output associated with the convolutional neural network.In certain embodiments, the medical imaging diagnosis component 106 candetermine one or more confidence scores for the classification and/orthe localization. For example, a first portion of the anatomical regionwith a greatest likelihood of a disease can be assigned a firstconfidence score, a second portion of the anatomical region with alesser degree of likelihood of a disease can be assigned a secondconfidence score, etc. A disease classified and/or localized by themedical imaging diagnosis component 106 can include, for example, a lungdisease, a heart disease, a tissue disease, a bone disease, a tumor, acancer, tuberculosis, cardiomegaly, hypoinflation of a lung, opacity ofa lung, hyperdistension, a spine degenerative disease, calcinosis, oranother type of disease associated with an anatomical region of apatient body. In an aspect, the medical imaging diagnosis component 106can determine a prediction for a disease associated with the medicalimaging data. For example, the medical imaging diagnosis component 106can determine a probability score for a disease associated with themedical imaging data (e.g., a first percentage value representinglikelihood of a negative prognosis for the disease and a second valuerepresenting a likelihood of a positive prognosis for the disease).

The visualization component 108 can generate deep learning data (e.g.,DEEP LEARNING DATA shown in FIG. 1) based on the classification and/orthe localization for the portion of the anatomical region. In anembodiment, the deep learning data can include a classification and/or alocation for one or more diseases located in the medical imaging data.In certain embodiments, the deep learning data can include probabilitydata indicative of a probability for one or more diseases being locatedin the medical imaging data. The probability data can be, for example, aprobability array of data values for one or more diseases being locatedin the medical imaging data. In another embodiment, the visualizationcomponent 108 can generate a multi-dimensional visualization associatedwith the classification and/or the localization for the portion of theanatomical region. The multi-dimensional visualization can be agraphical representation of the medical imaging data that shows aclassification and/or a location of one or more diseases with respect toa patient body. The visualization component 108 can also generate adisplay of the multi-dimensional visualization of the diagnosis providedby the medical imaging diagnosis component 106. For example, thevisualization component 108 can render a 2D visualization of the portionof the anatomical region on a user interface associated with a displayof a user device such as, but not limited to, a computing device, acomputer, a desktop computer, a laptop computer, a monitor device, asmart device, a smart phone, a mobile device, a handheld device, atablet, a portable computing device or another type of user deviceassociated with a display. In an aspect, the multi-dimensionalvisualization can include the deep learning data. The deep learning dataassociated with the multi-dimensional visualization can be indicative ofa visual representation of the classification and/or the localizationfor the portion of the anatomical region. The deep learning data canalso be rendered on the 3D model as one or more dynamic visual elements.In an aspect, the visualization component 108 can alter visualcharacteristics (e.g., color, size, hues, shading, etc.) of at least aportion of the deep learning data associated with the multi-dimensionalvisualization based on the classification and/or the localization forthe portion of the anatomical region. For example, the classificationand/or the localization for the portion of the anatomical region can bepresented as different visual characteristics (e.g., colors, sizes, huesor shades, etc.), based on a result of deep learning and/or medicalimaging diagnosis by the machine learning component 104 and/or themedical imaging diagnosis component 106. In another aspect, thevisualization component 108 can allow a user to zoom into or out withrespect to the deep learning data associated with the multi-dimensionalvisualization. For example, the visualization component 108 can allow auser to zoom into or out with respect to a classification and/or alocation of one or more diseases identified in the anatomical region ofthe patient body. As such, a user can view, analyze and/or interact withthe deep learning data associated with the multi-dimensionalvisualization.

It is to be appreciated that technical features of the deep learningcomponent 102 are highly technical in nature and not abstract ideas.Processing threads of the deep learning component 102 that processand/or analyze the medical imaging data, determine deep learning data,etc. cannot be performed by a human (e.g., are greater than thecapability of a single human mind). For example, the amount of themedical imaging data processed, the speed of processing of the medicalimaging data and/or the data types of the medical imaging data processedby the deep learning component 102 over a certain period of time can berespectively greater, faster and different than the amount, speed anddata type that can be processed by a single human mind over the sameperiod of time. Furthermore, the medical imaging data processed by thedeep learning component 102 can be one or more medical images generatedby sensors of a medical imaging device. Moreover, the deep learningcomponent 102 can be fully operational towards performing one or moreother functions (e.g., fully powered on, fully executed, etc.) whilealso processing the medical imaging data.

Referring now to FIG. 2, there is illustrated a non-limitingimplementation of a system 200 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 200 can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component 106, the visualizationcomponent 108, the processor 110 and/or the memory 112. The machinelearning component 104 can include a spring self-transfer learning (STL)network component 202.

The spring STL network component 202 can provide a weakly supervisedframework with self-transfer learning. For instance, a convolutionalneural network model employed by the spring STL network component 202can be pre-trained using a set of medical images formatted asimage-level labels (e.g., weak-labeled images) without locationinformation with respect to localization of one or more features of theset of medical images. The convolutional neural network model employedby the spring STL network component 202 can employ a plurality ofsequential downsampling and upsampling of the medical imaging datareceived by the spring STL network component 202. For example, theplurality of sequential downsampling and upsampling can be performed byshared spring convolutional layers that behave in a spring-like mannerThe shared spring convolutional layers can include convolutional layerfilters with various sizes. Furthermore, one or more convolutional layerfilters from the shared spring convolutional layers can be repeated. Inan aspect, the spring STL network component 202 can generate learnedmedical imaging output associated with the medical imaging data based onthe shared spring convolutional layers. The spring STL network component202 can additionally employ classification layers and/or localizationlayers to generate the learned medical imaging output. For instance, thelearned medical imaging output generated by the spring STL networkcomponent 202 can include one or more classifications for the medicalimaging data that is determined based on the shared spring convolutionallayers and the classification layers. Additionally or alternatively, thelearned medical imaging output generated by the spring STL networkcomponent 202 can include one or more localizations for the medicalimaging data that is determined based on the shared spring convolutionallayers and the localization layers.

Referring now to FIG. 3, there is illustrated a non-limitingimplementation of a system 300 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 300 can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component 106, the visualizationcomponent 108, the processor 110 and/or the memory 112. The machinelearning component 104 can include the spring STL network component 202.The spring STL network component 202 can include a spring convolutionallayers component 302.

The spring convolutional layers component 302 can execute the sharedspring convolutional layers. The shared spring convolutional layers canbe associated with a machine learning convolutional layer process. Theshared spring convolutional layers executed by the spring convolutionallayers component 302 can behave in a spring-like manner For example, theshared spring convolutional layers executed by the spring convolutionallayers component 302 can include convolutional layer filters withvarious sizes. Furthermore, one or more convolutional layer filters fromthe shared spring convolutional layers executed by the springconvolutional layers component 302 can be repeated. For instance, sharedspring convolutional layers executed by the spring convolutional layerscomponent 302 can include a first convolutional layer filter thatcomprises a first size, a second convolutional layer filter thatcomprises a second size that is different than the first size, a thirdconvolutional layer filter that comprises the first size associated withthe first convolutional layer filter, etc. In an aspect, springconvolutional layers component 302 can extract feature information fromthe medical imaging data using the shared spring convolutional layers.The feature information can include, for example, a set of data matrices(e.g., a set of feature maps) extracted from the medical imaging data. Asize of the set of data matrices can be smaller than a size of a datamatrix associated with the medical imaging data.

Referring now to FIG. 4, there is illustrated a non-limitingimplementation of a system 400 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 400 can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component 106, the visualizationcomponent 108, the processor 110 and/or the memory 112. The machinelearning component 104 can include the spring STL network component 202.The spring STL network component 202 can include the springconvolutional layers component 302 and/or a classification layerscomponent 402.

The classification layers component 402 can be a classifier thatperforms a classification process to classify the medical imaging data.The classification layers component 402 can employ a set ofclassification layers (e.g., a set of fully connected layers) to performthe classification process. For instance, the classification layerscomponent 402 can classify the medical imaging data into a class basedon the set of classification layers (e.g., the set of fully connectedlayers). The classification layers component 402 can also employ atraining dataset to facilitate the classification of the medical imagingdata. For instance, the classification layers component 402 can alsoclassify the medical imaging data into the class based on the trainingdataset. The training dataset can be generated during a training phasethat trains the convolutional neural network model employed by thespring convolutional layers component 302. In an embodiment, theclassification layers component 402 can employ an automaticclassification system and/or an automatic classification process tofacilitate analysis of the medical imaging data. For example, theclassification layers component 402 can employ a probabilistic and/orstatistical-based analysis (e.g., factoring into the analysis utilitiesand costs) to learn and/or generate inferences with respect to themedical imaging data. The classification layers component 402 canemploy, for example, a support vector machine (SVM) classifier to learnand/or generate inferences for medical imaging data. Additionally oralternatively, the classification layers component 402 can employ otherclassification techniques associated with Bayesian networks, decisiontrees and/or probabilistic classification models. Classifiers employedby the classification layers component 402 can be explicitly trained(e.g., via a generic training data) as well as implicitly trained (e.g.,via receiving extrinsic information). For example, with respect toSVM's, SVM's can be configured via a learning or training phase within aclassifier constructor and feature selection module. A classifier can bea function that maps an input attribute vector, x=(x1, x2, x3, x4, xn),to a confidence that the input belongs to a class—that is,f(x)=confidence(class).

Referring now to FIG. 5, there is illustrated a non-limitingimplementation of a system 500 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 500 can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component 106, the visualizationcomponent 108, the processor 110 and/or the memory 112. The machinelearning component 104 can include the spring STL network component 202.The spring STL network component 202 can include the springconvolutional layers component 302, the classification layers component402 and/or a localization layers component 502. The localization layerscomponent 502 can include an activation map component 504, a regularizercomponent 506 and/or a global pooling component 508.

The localization layers component 502 can be a localizer that performs alocalization process to localize one or more classifications of themedical imaging data. The localization layers component 502 can employ aset of localization layers to perform the localization process. Forinstance, the localization layers component 502 can localize aclassification of the medical imaging data based on the set oflocalization layers. In an embodiment, the activation map component 504can generate a set of activation maps. The set of activation maps can bescore maps for each class associated with the classification layerscomponent 402. For instance, a number of activation maps included in theset of activation maps can correspond to a number of classes determinedby the classification layers component 402. In another embodiment, theregularizer component 506 can be employed to reduce overfittingassociated with the set of activation maps. The regularizer component506 can perform a local pooling process that reduces dimensionality ofthe set of activation maps. For example, the regularizer component 506can include a local pooling layer to reduce singularity issues and/or toimprove localization for the localization layers component 502. In yetanother embodiment, the global pooling component 508 can perform aglobal pooling process that further reduces dimensionality of the set ofactivation maps. In an aspect, a size of a filter associated with theglobal pooling process can be larger than a size of a filter associatedwith the local pooling layer. In one example, a size of a filterassociated with the global pooling process can correspond to a size ofthe medical imaging data.

Referring now to FIG. 6, there is illustrated a non-limitingimplementation of a system 600 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 600 can include the deep learning component102, and the deep learning can include the machine learning component104, the medical imaging diagnosis component 106, the visualizationcomponent 108, the processor 110 and/or the memory 112. The machinelearning component 104 can include the spring STL network component 202.The spring STL network component 202 can include the springconvolutional layers component 302, the classification layers component402, the localization layers component 502 and/or a training component602. The localization layers component 502 can include the activationmap component 504, the regularizer component 506 and/or the globalpooling component 508.

The training component 602 can perform a training phase for a neuralnetwork model employed by the spring convolutional layers component 302.For example, the medical imaging data can be a set of medical images(e.g., a set of x-ray images, etc.) stored in a data store. Furthermore,the training component 602 can perform the training phase for a neuralnetwork model (e.g., a convolutional neural network model) based on theset of medical images stored in a data store to train the neural networkmodel. In an embodiment, training component 602 can train aconvolutional neural network (e.g., the neural network model) byperforming iterative sequential downsampling and upsampling of themedical imaging data associated with convolutional layers of theconvolutional neural network. In an aspect, the training component 602can generate a set of filter values for the convolutional neural network(e.g., the neural network model) based on the iterative sequentialdownsampling and upsampling of the medical imaging data. For example,the training component 602 can generate a set of as set of weights for aset of filters associated with the convolutional neural network (e.g.,the neural network model) based on the iterative sequential downsamplingand upsampling of the medical imaging data. In certain embodiments, thetraining component 602 can analyze the medical imaging data based on afirst convolutional layer filter that comprises a first size, analyzethe medical imaging database on a second convolutional layer filter thatcomprises a second size that is different than the first size, analyzethe medical imaging database on a third convolutional layer filter thatcomprises the first size associated with the first convolutional layerfilter, etc.

Referring now to FIG. 7, there is illustrated a non-limitingimplementation of a system 700 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 700 can include spring convolutional layers702, classification layers 704, localization layers 706, a weighted addphase 708 and/or a training phase 710. The localization layers 706 caninclude activation maps 712, a regularizer 714 and/or global pooling716.

The system 700 can be, for example, a spring self-transfer learningnetwork. The spring convolutional layers 702 can receive the medicalimaging data. In an embodiment, the spring convolutional layers 702 canbe executed by the spring convolutional layers component 302. The springconvolutional layers 702 can behave in a spring-like manner For example,spring convolutional layers 702 can include convolutional layer filterswith various sizes. Furthermore, one or more convolutional layer filtersfrom the spring convolutional layers 702 can be repeated. For instance,the spring convolutional layers 702 can include a first convolutionallayer filter that comprises a first size, a second convolutional layerfilter that comprises a second size that is different than the firstsize, a third convolutional layer filter that comprises the first sizeassociated with the first convolutional layer filter, etc. In an aspect,the spring convolutional layers 702 can extract feature information fromthe medical imaging data. The feature information can include, forexample, a set of data matrices (e.g., a set of feature maps) extractedfrom the medical imaging data. A size of the set of data matrices can besmaller than a size of a data matrix associated with the medical imagingdata.

The classification layers 704 can further process the featureinformation extracted from the spring convolutional layers 702. In anembodiment, the classification layers 704 can be executed by theclassification layers component 402. The classification layers 704 candetermine one or more classifications for the medical imaging data. Inan aspect, each CNN neuron in a first layer (e.g., a previous layer)from the classification layers 704 can be connected to each neuron in asecond layer (e.g., a next layer) from the classification layers 704. Inanother aspect, the one or more classifications for the medical imagingdata can be received by the weighted add phase 708.

The localization layers 706 can also further process the featureinformation extracted from the spring convolutional layers 702. In anembodiment, the localization layers 706 can be executed by thelocalization layers component 502. The localization layers 706 candetermine one or more localizations for the medical imaging data. In anaspect, the localization layers 706 can employ the activation maps 712,the regularizer 714 and/or the global pooling 716 to determine the oneor more localizations for the medical imaging data. In an embodiment,the activation maps 712 can be executed by the activation map component504, the regularizer 714 can correspond to the regularizer component506, and/or the global pooling 716 can be performed by the globalpooling component 508. The activation maps 712 can be score maps (e.g.,class activation maps) for each class associated with the springconvolutional layers 702. For instance, a number of the activation maps712 can correspond to a number of classes associated with the springconvolutional layers 702.

The regularizer 714 can reduce overfitting associated with theactivation maps 712. The regularizer 714 can be, for example, a localpooling process that reduces dimensionality of the activation maps 712.For example, the regularizer 714 can include a local pooling layer thatreduces dimensionality of the activation maps 712. The global pooling716 can be a global pooling process that further reduces dimensionalityof the activation maps 712. In an aspect, a size of a filter associatedwith the global pooling 716 can be larger than a size of a filterassociated with the regularizer 714. In one example, a size of a filterassociated with the global pooling 716 can correspond to a size of themedical imaging data received by the spring convolutional layers 702. Assuch, the regularizer 714 (e.g., a local pooling layer associated withthe regularizer 714) can be performed prior to the global pooling 716with respect to the activation maps 712 to, for example, overcome one ormore singularity issues and/or to improve localization by thelocalization layers 706. In an aspect, the one or more localizations forthe medical imaging data can be received by the weighted add phase 708.The weighted add phase 708 can combine the one or more classificationsand the one or more localizations to generate the learned medicalimaging output. For example, the learned medical imaging output canprovide a classification and/or a location for one or more featuresassociated with the medical imaging data.

In certain embodiments, the system 700 can employ the training phase710. The training phase 710 can perform a training phase for a neuralnetwork model employed by the spring convolutional layers 702. Forexample, the medical imaging data can be a set of medical images (e.g.,a set of x-ray images, etc.) stored in a data store. Furthermore, thetraining phase 710 can perform the training phase for a neural networkmodel (e.g., a convolutional neural network model) based on the set ofmedical images stored in a data store to train the neural network model.In an embodiment, training phase 710 can train a convolutional neuralnetwork (e.g., the neural network model) by performing iterativesequential downsampling and upsampling of the medical imaging dataassociated with convolutional layers of the convolutional neuralnetwork. In an aspect, the training phase 710 can generate a set offilter values for the convolutional neural network (e.g., the neuralnetwork model) based on the iterative sequential downsampling andupsampling of the medical imaging data. For example, the training phase710 can generate a set of as set of weights for a set of filtersassociated with the convolutional neural network (e.g., the neuralnetwork model) based on the iterative sequential downsampling andupsampling of the medical imaging data. In an embodiment, the trainingphase 710 can analyze the medical imaging data based on a firstconvolutional layer filter that comprises a first size, analyze themedical imaging database on a second convolutional layer filter thatcomprises a second size that is different than the first size, analyzethe medical imaging database on a third convolutional layer filter thatcomprises the first size associated with the first convolutional layerfilter, etc.

Referring now to FIG. 8, there is illustrated a non-limitingimplementation of a system 800 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 800 can be associated with an inferencephase for a spring self-transfer learning network. The system 800 caninclude medical imaging data 802 that is received by a pooling layerprocess 804. In an embodiment, the medical imaging data 802 cancorrespond to the medical imaging data received by the deep learningcomponent 102. The medical imaging data 802 can be, for example,two-dimensional medical imaging data and/or three-dimensional medicalimaging data generated by a medical imaging device. For instance, themedical imaging data 802 can be electromagnetic radiation imagerycaptured via a set of sensors (e.g., a set of sensors associated with amedical imaging device). In one example, the medical imaging data 802can be an x-ray image. The pooling layer process 804 can format themedical imaging data 802 for processing by the spring convolutionallayer process 806. For example, the pooling layer process 804 canconvert the medical imaging data 802 into a data matrix with aparticular size. In certain embodiments, the pooling layer process 804can reduce dimensionality of the medical imaging data 802. For example,the pooling layer process 804 can reduce the particular size of the datamatrix. The pooling layer process 804 can be followed by a springconvolutional layer process 806. The spring convolutional layer process806 can be a machine learning convolutional layer process. Furthermore,the spring convolutional layer process 806 can be, for example, asequential convolutional layer process that behaves in a spring-likemanner The spring convolutional layer process 806 can include aplurality of sequential downsampling and upsampling of the medicalimaging data 802. For example, the plurality of sequential downsamplingand upsampling of the spring convolutional layer process 806 can beperformed by shared spring convolutional layers that behave in aspring-like manner The shared spring convolutional layers of the springconvolutional layer process 806 can include convolutional layer filterswith various sizes. Furthermore, one or more convolutional layer filtersfrom the shared spring convolutional layers of the spring convolutionallayer process 806 can be repeated.

The spring convolutional layer process 806 can be followed by a fullyconnected layer process 808 implemented in parallel to an activation mapprocess 810. The fully connected layer process 808 can be a machinelearning classification process that classifies the medical imaging data802. In an aspect, the fully connected layer process 808 can determineone or more classes for the medical imaging data 802. The activation mapprocess 810 can generate a set of activation maps for the medicalimaging data 802. For example, the set of activation maps generated bythe activation map process 810 can be a set of score maps associatedwith the one or more classes determined by the fully connected layerprocess 808. In an aspect, a number of activation maps included in theset of activation maps can correspond to a number of classes determinedby the fully connected layer process 808. The activation map process 810can be followed by a pooling layer process 812. The pooling layerprocess 812 can reduce dimensionality of the set of activation mapsgenerated by the activation map process 810.

A heat map 814 can be generated following the pooling layer process 812.The heat map 814 can include one or more localizations to localize oneor more classifications of the medical imaging data 802. For example,the heat map 814 can be a graphical representation of data generated bythe spring convolutional layer process 806 and/or the activation mapprocess 810. The data generated by the spring convolutional layerprocess 806 and/or the activation map process 810 can be represented asdifferent colors based on a value of the data. For example, one or moredata values that satisfy a first defined criterion (e.g., one or moredata values that represents a high degree of localization) can berepresented as a red color, one or more data values that satisfy asecond defined criterion (e.g., one or more data values that representsa medium degree of localization) can be represented as a green color,one or more data values that satisfy a third defined criterion (e.g.,one or more data values that represents a low degree of localization)can be represented as a green color, etc. The heat map 814 can becombined with the medical imaging data 802 to generate amulti-dimensional visualization 816. The pooling layer process 812 canalso be followed by a global pooling layer process 818. The globalpooling layer process 818 can further alter dimensionality of the of theset of activation maps generated by the activation map process 810. Forexample, the global pooling layer process 818 alter dimensionality ofthe of the set of activation maps generated by the activation mapprocess 810 to correspond to dimensionality of the medical imaging data802. The global pooling layer process 818 can be followed by a weightedadd process 820. The fully connected layer process 808 can also befollowed by the weighted add process 820. The weighted add process 820can employ information from the spring convolutional layer process 806(e.g., one or more localizations) and the fully connected layer process808 (e.g., one or more classifications) to generate output probabilitydata.

Referring now to FIG. 9, there is illustrated a non-limitingimplementation of a system 900 in accordance with various aspects andimplementations of this disclosure. Repetitive description of likeelements employed in other embodiments described herein is omitted forsake of brevity. The system 900 can be associated with sequentialupsampling and downsampling for a spring self-transfer learning network.In an embodiment, the system 900 can be associated with the springconvolutional layers component 302.

The system 900 can include a convolutional layer 902. The convolutionallayer 902 can be a first convolutional layer of a convolutional neuralnetwork that processes medical imaging data. Furthermore, theconvolutional layer 902 can be associated with a first filter size. Theconvolutional layer 902 can be followed by a pooling layer (down) 904.The pooling layer (down) 904 can be associated with downsampling. Forinstance, the pooling layer (down) 904 can reduce dimensionality of datagenerated by the convolutional layer 902. In one example, the poolinglayer (down) 904 can reduce dimensionality of a feature map for medicalimaging data processed by the convolutional layer 902. The pooling layer(down) 904 can be followed by a convolutional layer 906. Theconvolutional layer 906 can be a second convolutional layer of theconvolutional neural network that processes the medical imaging data.Furthermore, the convolutional layer 906 can be associated with a secondfilter size that is different than the first filter size associated withthe convolutional layer 902. For example, the second filter sizeassociated with the convolutional layer 906 can be smaller than thefirst filter size associated with the convolutional layer 902. Theconvolutional layer 906 can be followed by a pooling layer (down) 908.The pooling layer (down) 908 can be associated with downsampling. Forinstance, the pooling layer (down) 908 can reduce dimensionality of datagenerated by the convolutional layer 906. In one example, the poolinglayer (down) 908 can reduce dimensionality of a feature map for medicalimaging data processed by the convolutional layer 906. The pooling layer(down) 908 can be followed by a convolutional layer (not shown), which,in turn, can be followed by a pooling layer (up) 910. However, incertain embodiments, the pooling layer (down) 910 can be followed by oneor more other convolutional layers and/or one or more other poolinglayers (down) prior to the pooling layer (up) 910 to further processmedical imaging data with different filter sizes and/or furtherreduction to dimensionality of data. The pooling layer (up) 910 can beassociated with upsampling. For instance, the pooling layer (up) 910 canincrease dimensionality of data generated by one or more convolutionallayers. In one example, the pooling layer (up) 910 can increasedimensionality of a feature map for medical imaging data processed byone or more convolutional layers. The pooling layer (up) 910 can befollowed by a convolutional layer 912. The convolutional layer 912 canbe, for example, a third convolutional layer of the convolutional neuralnetwork that processes the medical imaging data. Furthermore, theconvolutional layer 912 can be associated with the second filter sizeassociated with the convolutional layer 906.

The convolutional layer 912 can be followed by a pooling layer (up) 914.The pooling layer (up) 914 can be associated with upsampling. Forinstance, the pooling layer (up) 914 can increase dimensionality of datagenerated by the convolutional layer 912. In one example, the poolinglayer (up) 914 can increase dimensionality of a feature map for medicalimaging data processed by the convolutional layer 912. The pooling layer(up) 914 can be followed by a convolutional layer 916. The convolutionallayer 916 can be, for example, a fourth convolutional layer of theconvolutional neural network that processes the medical imaging data.Furthermore, the convolutional layer 916 can be associated with thefirst filter size associated with the convolutional layer 912. As such,the system 900 can behave similar to functionality of a spring where afilter size for one or more convolutional layers are repeated whileprocessing medical imaging data via a convolutional neural network.

FIG. 10 illustrates an example multi-dimensional visualization 1000, inaccordance with various aspects and implementations described herein.The multi-dimensional visualization 1000 can, for example, display amedical imaging diagnosis for a patient. For example, themulti-dimensional visualization 1000 can display one or moreclassifications and/or one or more localizations for one or morediseases identified in medical imaging data. In an aspect, themulti-dimensional visualization 1000 can include localization data 1002for a medical imaging diagnosis. The localization data 1002 can be apredicted location for a disease associated with medical imaging dataprocessed by the machine learning component 104 and/or the medicalimaging diagnosis component 106. Visual characteristics (e.g., a color,a size, hues, shading, etc.) of the localization data 1002 can bedynamic based on information provided by the machine learning component104 and/or the medical imaging diagnosis component 106. For instance, afirst portion of the localization data 1002 can comprise a first visualcharacteristic, a second portion of the localization data 1002 cancomprise a second visual characteristic, a third portion of thelocalization data 1002 can comprise a third visual characteristic, etc.In an embodiment, a display environment associated with themulti-dimensional visualization 1000 can include a heat bar 1004. Theheat bar 1004 can include a set of colors that correspond to differentvalues for the localization data 1002. For example, a first color (e.g.,a color red) in the heat bar 1004 can correspond to a first value forthe localization data 1002, a second color (e.g., a color green) in theheat bar 1004 can correspond to a second value for the localization data1002, a third color (e.g., a color blue) in the heat bar 1004 cancorrespond to a third value for the localization data 1002, etc.

FIG. 11 illustrates an example user interface 1100, in accordance withvarious aspects and implementations described herein. The user interface1100 can be a display environment for medical imaging data and/or deeplearning data associated with medical imaging data. The user interface1100 can include medical imaging data 1102. In one embodiment, themedical imaging data 1102 can be displayed as a multi-dimensionalvisualization that presents a medical imaging diagnosis for a patient.For example, in certain embodiments, the medical imaging data 1102 canbe displayed as a multi-dimensional visualization that presents one ormore classifications and/or one or more localizations for one or morediseases identified in medical imaging data 1102. In certainembodiments, the medical imaging data 1102 can be displayed as amulti-dimensional visualization that presents localization data for amedical imaging diagnosis. In another embodiment, the user interface1100 can include a heat bar 1104. The heat bar 1104 can include a set ofcolors that correspond to different values for the localization data.The user interface 1100 can also include a prediction section 1106 topresent one or more predictions associated with the medical imaging data1102. The prediction section 1106 can include a patient name 1108 for apatient (e.g., a patient body) associated with the medical imaging data1102. The prediction section 1106 can also include a condition portion1110 and a prediction portion 1112. The condition portion 1110 caninclude one or more conditions such as, for example, a tuberculosiscondition 1110 a, a lateral view condition 1110 b, a cardiomegalycondition 1110 c, an opacity/lung condition 1110 d, a lung/hypoinflationcondition 1110 e, a hyperdistention condition 1110 f, a spinedegenerative condition 1110 g, a calcinosis condition 1110 h and/oranother type of condition. The prediction portion 1112 can includecorresponding predictions 1112 a-h for the conditions included in thecondition portion 1110. For example, the prediction 1112 a can include aprediction for the medical imaging data 1102 being associated withtuberculosis (e.g., a 38.42% chance of a negative prognosis fortuberculosis and a 61.58% chance of a positive prognosis fortuberculosis). In another example, the prediction 1112 h can include aprediction for the medical imaging data 1102 being associated withcalcinosis (e.g., a 40.99% chance of a negative prognosis for calcinosisand a 59.01% chance of a positive prognosis for calcinosis). In certainembodiments, the prediction section 1106 can also include a patient age1114, a patient gender 1116 and/or other information regarding a patientassociated with the patient name 1108. As such, in certain embodiments,the medical imaging data 1102 can be associated with multiple diseases.Furthermore, multiple inferencing models can be employed and aggregatedas deep learning data shown in the user interface 1100.

FIGS. 12-13 illustrate methodologies and/or flow diagrams in accordancewith the disclosed subject matter. For simplicity of explanation, themethodologies are depicted and described as a series of acts. It is tobe understood and appreciated that the subject innovation is not limitedby the acts illustrated and/or by the order of acts, for example actscan occur in various orders and/or concurrently, and with other acts notpresented and described herein. Furthermore, not all illustrated actsmay be required to implement the methodologies in accordance with thedisclosed subject matter. In addition, those skilled in the art willunderstand and appreciate that the methodologies could alternatively berepresented as a series of interrelated states via a state diagram orevents. Additionally, it should be further appreciated that themethodologies disclosed hereinafter and throughout this specificationare capable of being stored on an article of manufacture to facilitatetransporting and transferring such methodologies to computers. The termarticle of manufacture, as used herein, is intended to encompass acomputer program accessible from any computer-readable device or storagemedia.

Referring to FIG. 12, there is illustrated a non-limiting implementationof a methodology 1200 for facilitating a deep convolutional neuralnetwork with self-transfer learning, according to an aspect of thesubject innovation. At 1202, medical imaging data for a patient body isreceived by a system comprising a processor (e.g., by machine learningcomponent 104). The medical imaging data can be, for example, a medicalimage such as electromagnetic radiation imagery, an x-ray image, a CTscan image, another type of medical image, etc. In an embodiment, themedical imaging data can be electromagnetic radiation imagery capturedvia a set of sensors (e.g., a set of sensors associated with a medicalimaging device).

At 1204, it is determined whether new medical imaging data is available.If yes, methodology 1200 returns to 1202. If no, methodology 1200proceeds to 1206.

At 1206, iterative sequential downsampling and upsampling of the medicalimaging data associated with convolutional layers of a convolutionalneural network is performed, by the system (e.g., by machine learningcomponent 104), to generate learned medical imaging output regarding thepatient body. The iterative sequential downsampling and upsampling ofthe medical imaging data can behave in a spring-like manner For example,the iterative sequential downsampling and upsampling of the medicalimaging data can be associated with convolutional layer filters withvarious sizes. One or more convolutional layer filters can be repeated.For instance, the iterative sequential downsampling and upsampling ofthe medical imaging data can include a first convolutional layer filterthat comprises a first size, a second convolutional layer filter thatcomprises a second size that is different than the first size, a thirdconvolutional layer filter that comprises the first size associated withthe first convolutional layer filter, etc. In certain embodiments, theperforming the iterative sequential downsampling and upsampling of themedical imaging data can include analyzing the medical imaging databased on a first filter that comprises a first size, analyzing themedical imaging data based on a second filter that comprises a secondsize that is different than the first size, analyzing the medicalimaging data based on a third filter that comprises the first sizeassociated with the first filter, etc. In another embodiment, theperforming the iterative sequential downsampling and upsampling of themedical imaging data can include generating the learned medical imagingoutput based on a first convolutional layer process associated withdownsampling of the medical imaging data and a second convolutionallayer process associated with upsampling of the medical imaging data. Inan aspect, the performing the iterative sequential downsampling andupsampling of the medical imaging data can be associated with automatedfeature detection for the medical imaging data.

At 1208, a disease for a portion of the patient body classifying, by thesystem (e.g., by medical imaging diagnosis component 106), based on thelearned medical imaging output associated with the convolutional neuralnetwork. A disease can include, for example, a lung disease, a heartdisease, a tissue disease, a bone disease, a tumor, a cancer,tuberculosis, cardiomegaly, hypoinflation of a lung, opacity of a lung,hyperdistension, a spine degenerative disease, calcinosis, or anothertype of disease associated with an anatomical region of a patient body.In an embodiment, a prediction for the disease can be determined. Forexample, a probability score for the disease can be determined (e.g., afirst percentage value representing likelihood of a negative prognosisfor the disease and a second value representing a likelihood of apositive prognosis for the disease can be determined).

At 1210, a multi-dimensional visualization associated with theclassifying of the disease for the portion of the patient body isgenerated by the system (e.g., by visualization component 108. Themulti-dimensional visualization can be a graphical representation of themedical imaging data that shows a classification and/or a location ofone or more diseases with respect to a patient body. In an aspect,visual characteristics (e.g., color, size, hues, shading, etc.) of atleast a portion of the multi-dimensional visualization can be alteredbased on the classification and/or a location of one or more diseaseswith respect to a patient body.

In certain embodiments, the methodology 1200 can additionally includeperforming, by the system, a local pooling process for an activation mapassociated with a convolutional layer of the convolutional neuralnetwork prior to performing a global pooling process associated with theconvolutional neural network. In certain embodiments, the methodology1200 can additionally or alternatively include generating, by thesystem, the learned medical imaging output based on a class activationmapping process that applies a set of weights to a set of heat mapsassociated with the medical imaging data. Furthermore, in certainembodiments, the methodology 1200 can additionally or alternativelyinclude merging, by the system, a set of classifier layers associatedwith the convolutional neural network and a set of activation mapsassociated with the convolutional neural network to generate the learnedmedical imaging output.

Referring to FIG. 13, there is illustrated a non-limiting implementationof a methodology 1300 for facilitating training of a deep convolutionalneural network with self-transfer learning, according to an aspect ofthe subject innovation. At 1302, medical imaging data that comprises aset of medical images is received by a system comprising a processor(e.g., by machine learning component 104). The medical imaging data canbe, for example, a set of medical images stored in a data store. In oneexample, the set of medical images can be a set of x-ray images and/or aset of CT scan images.

At 1304, it is determined whether new medical imaging data is available.If yes, methodology 1300 returns to 1302. If no, methodology 1300proceeds to 1306.

At 1306, a convolutional neural network is training, by the system(e.g., by training component 602), by performing iterative sequentialdownsampling and upsampling of the medical imaging data associated withconvolutional layers of the convolutional neural network. The iterativesequential downsampling and upsampling of the medical imaging data canbehave in a spring-like manner For example, the iterative sequentialdownsampling and upsampling of the medical imaging data can beassociated with convolutional layer filters with various sizes. One ormore convolutional layer filters can be repeated. For instance, theiterative sequential downsampling and upsampling of the medical imagingdata can include a first convolutional layer filter that comprises afirst size, a second convolutional layer filter that comprises a secondsize that is different than the first size, a third convolutional layerfilter that comprises the first size associated with the firstconvolutional layer filter, etc.

At 1308, a set of filter values for the convolutional neural network isgenerated, by the system (e.g., by training component 602), based on theiterative sequential downsampling and upsampling of the medical imagingdata. For example, a set of weights for a set of filters associated withthe convolutional neural network can be generated based on the iterativesequential downsampling and upsampling of the medical imaging data.

The aforementioned systems and/or devices have been described withrespect to interaction between several components. It should beappreciated that such systems and components can include thosecomponents or sub-components specified therein, some of the specifiedcomponents or sub-components, and/or additional components.Sub-components could also be implemented as components communicativelycoupled to other components rather than included within parentcomponents. Further yet, one or more components and/or sub-componentsmay be combined into a single component providing aggregatefunctionality. The components may also interact with one or more othercomponents not specifically described herein for the sake of brevity,but known by those of skill in the art.

In order to provide a context for the various aspects of the disclosedsubject matter, FIGS. 14 and 15 as well as the following discussion areintended to provide a brief, general description of a suitableenvironment in which the various aspects of the disclosed subject mattermay be implemented.

With reference to FIG. 14, a suitable environment 1400 for implementingvarious aspects of this disclosure includes a computer 1412. Thecomputer 1412 includes a processing unit 1414, a system memory 1416, anda system bus 1418. The system bus 1418 couples system componentsincluding, but not limited to, the system memory 1416 to the processingunit 1414. The processing unit 1414 can be any of various availableprocessors. Dual microprocessors and other multiprocessor architecturesalso can be employed as the processing unit 1414.

The system bus 1418 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), Firewire (IEEE 1394), and SmallComputer Systems Interface (SCSI).

The system memory 1416 includes volatile memory 1420 and nonvolatilememory 1422. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer1412, such as during start-up, is stored in nonvolatile memory 1422. Byway of illustration, and not limitation, nonvolatile memory 1422 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable programmable ROM(EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g.,ferroelectric RAM (FeRAM). Volatile memory 1420 includes random accessmemory (RAM), which acts as external cache memory. By way ofillustration and not limitation, RAM is available in many forms such asstatic RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), doubledata rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM(SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM),and Rambus dynamic RAM.

Computer 1412 also includes removable/non-removable,volatile/non-volatile computer storage media. FIG. 14 illustrates, forexample, a disk storage 1424. Disk storage 1424 includes, but is notlimited to, devices like a magnetic disk drive, floppy disk drive, tapedrive, Jaz drive, Zip drive, LS-100 drive, flash memory card, or memorystick. The disk storage 1424 also can include storage media separatelyor in combination with other storage media including, but not limitedto, an optical disk drive such as a compact disk ROM device (CD-ROM), CDrecordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or adigital versatile disk ROM drive (DVD-ROM). To facilitate connection ofthe disk storage devices 1424 to the system bus 1418, a removable ornon-removable interface is typically used, such as interface 1426.

FIG. 14 also depicts software that acts as an intermediary between usersand the basic computer resources described in the suitable operatingenvironment 1400. Such software includes, for example, an operatingsystem 1428. Operating system 1428, which can be stored on disk storage1424, acts to control and allocate resources of the computer system1412. System applications 1430 take advantage of the management ofresources by operating system 1428 through program modules 1432 andprogram data 1434, e.g., stored either in system memory 1416 or on diskstorage 1424. It is to be appreciated that this disclosure can beimplemented with various operating systems or combinations of operatingsystems.

A user enters commands or information into the computer 1412 throughinput device(s) 1436. Input devices 1436 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1414through the system bus 1418 via interface port(s) 1438. Interfaceport(s) 1438 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1440 usesome of the same type of ports as input device(s) 1436. Thus, forexample, a USB port may be used to provide input to computer 1412, andto output information from computer 1412 to an output device 1440.Output adapter 1442 is provided to illustrate that there are some outputdevices 1440 like monitors, speakers, and printers, among other outputdevices 1440, which require special adapters. The output adapters 1442include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1440and the system bus 1418. It should be noted that other devices and/orsystems of devices provide both input and output capabilities such asremote computer(s) 1444.

Computer 1412 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1444. The remote computer(s) 1444 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device or other common network node and the like, and typicallyincludes many or all of the elements described relative to computer1412. For purposes of brevity, only a memory storage device 1446 isillustrated with remote computer(s) 1444. Remote computer(s) 1444 islogically connected to computer 1412 through a network interface 1448and then physically connected via communication connection 1450. Networkinterface 1448 encompasses wire and/or wireless communication networkssuch as local-area networks (LAN), wide-area networks (WAN), cellularnetworks, etc. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL).

Communication connection(s) 1450 refers to the hardware/softwareemployed to connect the network interface 1448 to the bus 1418. Whilecommunication connection 1450 is shown for illustrative clarity insidecomputer 1412, it can also be external to computer 1412. Thehardware/software necessary for connection to the network interface 1448includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and Ethernet cards.

FIG. 15 is a schematic block diagram of a sample-computing environment1500 with which the subject matter of this disclosure can interact. Thesystem 1500 includes one or more client(s) 1510. The client(s) 1510 canbe hardware and/or software (e.g., threads, processes, computingdevices). The system 1500 also includes one or more server(s) 1530.Thus, system 1500 can correspond to a two-tier client server model or amulti-tier model (e.g., client, middle tier server, data server),amongst other models. The server(s) 1530 can also be hardware and/orsoftware (e.g., threads, processes, computing devices). The servers 1530can house threads to perform transformations by employing thisdisclosure, for example. One possible communication between a client1510 and a server 1530 may be in the form of a data packet transmittedbetween two or more computer processes.

The system 1500 includes a communication framework 1550 that can beemployed to facilitate communications between the client(s) 1510 and theserver(s) 1530. The client(s) 1510 are operatively connected to one ormore client data store(s) 1520 that can be employed to store informationlocal to the client(s) 1510. Similarly, the server(s) 1530 areoperatively connected to one or more server data store(s) 1540 that canbe employed to store information local to the servers 1530.

It is to be noted that aspects or features of this disclosure can beexploited in substantially any wireless telecommunication or radiotechnology, e.g., Wi-Fi; Bluetooth; Worldwide Interoperability forMicrowave Access (WiMAX); Enhanced General Packet Radio Service(Enhanced GPRS); Third Generation Partnership Project (3GPP) Long TermEvolution (LTE); Third Generation Partnership Project 2 (3GPP2) UltraMobile Broadband (UMB); 3GPP Universal Mobile Telecommunication System(UMTS); High Speed Packet Access (HSPA); High Speed Downlink PacketAccess (HSDPA); High Speed Uplink Packet Access (HSUPA); GSM (GlobalSystem for Mobile Communications) EDGE (Enhanced Data Rates for GSMEvolution) Radio Access Network (GERAN); UMTS Terrestrial Radio AccessNetwork (UTRAN); LTE Advanced (LTE-A); etc. Additionally, some or all ofthe aspects described herein can be exploited in legacytelecommunication technologies, e.g., GSM. In addition, mobile as wellnon-mobile networks (e.g., the Internet, data service network such asinternet protocol television (IPTV), etc.) can exploit aspects orfeatures described herein.

While the subject matter has been described above in the general contextof computer-executable instructions of a computer program that runs on acomputer and/or computers, those skilled in the art will recognize thatthis disclosure also can or may be implemented in combination with otherprogram modules. Generally, program modules include routines, programs,components, data structures, etc. that perform particular tasks and/orimplement particular abstract data types. Moreover, those skilled in theart will appreciate that the inventive methods may be practiced withother computer system configurations, including single-processor ormultiprocessor computer systems, mini-computing devices, mainframecomputers, as well as personal computers, hand-held computing devices(e.g., PDA, phone), microprocessor-based or programmable consumer orindustrial electronics, and the like. The illustrated aspects may alsobe practiced in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. However, some, if not all aspects of thisdisclosure can be practiced on stand-alone computers. In a distributedcomputing environment, program modules may be located in both local andremote memory storage devices.

As used in this application, the terms “component,” “system,”“platform,” “interface,” and the like, can refer to and/or can include acomputer-related entity or an entity related to an operational machinewith one or more specific functionalities. The entities disclosed hereincan be either hardware, a combination of hardware and software,software, or software in execution. For example, a component may be, butis not limited to being, a process running on a processor, a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on aserver and the server can be a component. One or more components mayreside within a process and/or thread of execution and a component maybe localized on one computer and/or distributed between two or morecomputers.

In another example, respective components can execute from variouscomputer readable media having various data structures stored thereon.The components may communicate via local and/or remote processes such asin accordance with a signal having one or more data packets (e.g., datafrom one component interacting with another component in a local system,distributed system, and/or across a network such as the Internet withother systems via the signal). As another example, a component can be anapparatus with specific functionality provided by mechanical partsoperated by electric or electronic circuitry, which is operated by asoftware or firmware application executed by a processor. In such acase, the processor can be internal or external to the apparatus and canexecute at least a part of the software or firmware application. As yetanother example, a component can be an apparatus that provides specificfunctionality through electronic components without mechanical parts,wherein the electronic components can include a processor or other meansto execute software or firmware that confers at least in part thefunctionality of the electronic components. In an aspect, a componentcan emulate an electronic component via a virtual machine, e.g., withina cloud computing system.

In addition, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom context, “X employs A or B” is intended to mean any of the naturalinclusive permutations. That is, if X employs A; X employs B; or Xemploys both A and B, then “X employs A or B” is satisfied under any ofthe foregoing instances. Moreover, articles “a” and “an” as used in thesubject specification and annexed drawings should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form.

As used herein, the terms “example” and/or “exemplary” are utilized tomean serving as an example, instance, or illustration. For the avoidanceof doubt, the subject matter disclosed herein is not limited by suchexamples. In addition, any aspect or design described herein as an“example” and/or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs, nor is it meantto preclude equivalent exemplary structures and techniques known tothose of ordinary skill in the art.

Various aspects or features described herein can be implemented as amethod, apparatus, system, or article of manufacture using standardprogramming or engineering techniques. In addition, various aspects orfeatures disclosed in this disclosure can be realized through programmodules that implement at least one or more of the methods disclosedherein, the program modules being stored in a memory and executed by atleast a processor. Other combinations of hardware and software orhardware and firmware can enable or implement aspects described herein,including a disclosed method(s). The term “article of manufacture” asused herein can encompass a computer program accessible from anycomputer-readable device, carrier, or storage media. For example,computer readable storage media can include but are not limited tomagnetic storage devices (e.g., hard disk, floppy disk, magnetic strips. . . ), optical discs (e.g., compact disc (CD), digital versatile disc(DVD), blu-ray disc (BD) . . . ), smart cards, and flash memory devices(e.g., card, stick, key drive . . . ), or the like.

As it is employed in the subject specification, the term “processor” canrefer to substantially any computing processing unit or devicecomprising, but not limited to, single-core processors;single-processors with software multithread execution capability;multi-core processors; multi-core processors with software multithreadexecution capability; multi-core processors with hardware multithreadtechnology; parallel platforms; and parallel platforms with distributedshared memory. Additionally, a processor can refer to an integratedcircuit, an application specific integrated circuit (ASIC), a digitalsignal processor (DSP), a field programmable gate array (FPGA), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Further, processors can exploit nano-scalearchitectures such as, but not limited to, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of user equipment. A processor may also beimplemented as a combination of computing processing units.

In this disclosure, terms such as “store,” “storage,” “data store,” datastorage,” “database,” and substantially any other information storagecomponent relevant to operation and functionality of a component areutilized to refer to “memory components,” entities embodied in a“memory,” or components comprising a memory. It is to be appreciatedthat memory and/or memory components described herein can be eithervolatile memory or nonvolatile memory, or can include both volatile andnonvolatile memory.

By way of illustration, and not limitation, nonvolatile memory caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable ROM (EEPROM), flashmemory, or nonvolatile random access memory (RAM) (e.g., ferroelectricRAM (FeRAM). Volatile memory can include RAM, which can act as externalcache memory, for example. By way of illustration and not limitation,RAM is available in many forms such as synchronous RAM (SRAM), dynamicRAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct RambusRAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM(RDRAM). Additionally, the disclosed memory components of systems ormethods herein are intended to include, without being limited toincluding, these and any other suitable types of memory.

It is to be appreciated and understood that components, as describedwith regard to a particular system or method, can include the same orsimilar functionality as respective components (e.g., respectively namedcomponents or similarly named components) as described with regard toother systems or methods disclosed herein.

What has been described above includes examples of systems and methodsthat provide advantages of this disclosure. It is, of course, notpossible to describe every conceivable combination of components ormethods for purposes of describing this disclosure, but one of ordinaryskill in the art may recognize that many further combinations andpermutations of this disclosure are possible. Furthermore, to the extentthat the terms “includes,” “has,” “possesses,” and the like are used inthe detailed description, claims, appendices and drawings such terms areintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim.

What is claimed is:
 1. A convolutional neural network system,comprising: a memory that stores computer executable components; aprocessor that executes computer executable components stored in thememory, wherein the computer executable components comprise: a machinelearning component that generates learned medical imaging outputregarding an anatomical region based on a convolutional neural networkthat receives medical imaging data, wherein the machine learningcomponent performs a plurality of sequential downsampling and upsamplingof the medical imaging data associated with convolutional layers of theconvolutional neural network; a medical imaging diagnosis component thatdetermines a classification and an associated localization for a portionof the anatomical region based on the learned medical imaging outputassociated with the convolutional neural network; and a visualizationcomponent that generates a multi-dimensional visualization associatedwith the classification and the localization for the portion of theanatomical region.
 2. The convolutional neural network system of claim1, wherein the machine learning component analyzes the medical imagingdata based on a first convolutional layer filter that comprises a firstsize, a second convolutional layer filter that comprises a second sizethat is different than the first size, and a third convolutional layerfilter that comprises the first size associated with the firstconvolutional layer filter.
 3. The convolutional neural network systemof claim 1, wherein the machine learning component performs a localpooling process for an activation map associated with a convolutionallayer of the convolutional neural network prior to performing a globalpooling process associated with the convolutional neural network.
 4. Theconvolutional neural network system of claim 1, wherein the machinelearning component generates the learned medical imaging output based ona first convolutional layer process associated with sequentialdownsampling of the medical imaging data and a second convolutionallayer process associated with sequential upsampling of the medicalimaging data.
 5. The convolutional neural network system of claim 1,wherein the machine learning component generates the learned medicalimaging output based on a class activation mapping process that appliesa set of weights to a set of heat maps associated with the medicalimaging data.
 6. The convolutional neural network system of claim 1,wherein the machine learning component merges a set of classifier layersassociated with the convolutional neural network and a set of activationmaps associated with the convolutional neural network to generate thelearned medical imaging output.
 7. The convolutional neural networksystem of claim 1, wherein the machine learning component processes themedical imaging data based on one or more regularization techniques toclassify one or more portions of the medical imaging data.
 8. Theconvolutional neural network system of claim 1, wherein the machinelearning component employs a first portion of the medical imaging datafor training associated with the convolutional neural network, a secondportion of the medical imaging data for validation associated with theconvolutional neural network, and a third portion of the medical imagingdata for testing associated with the convolutional neural network. 9.The convolutional neural network system of claim 1, wherein the machinelearning component randomly selects a set of medical images from atraining set associated with the medical imaging data for dataaugmentation associated with the medical imaging data.
 10. Theconvolutional neural network system of claim 9, wherein the machinelearning component modifies an orientation of the set of medical imagesfor the data augmentation associated with the medical imaging data. 11.The convolutional neural network system of claim 9, wherein the machinelearning component modifies the set of medical images through at leastone affine transformation for the data augmentation associated with themedical imaging data.
 12. A method, comprising: receiving, by a systemcomprising a processor, medical imaging data for a patient body;performing, by the system, iterative sequential downsampling andupsampling of the medical imaging data associated with convolutionallayers of a convolutional neural network to generate learned medicalimaging output regarding the patient body; classifying, by the system, adisease for a portion of the patient body based on the learned medicalimaging output associated with the convolutional neural network; andgenerating, by the system, a multi-dimensional visualization associatedwith the classifying of the disease for the portion of the patient body.13. The method of claim 12, wherein the performing the iterativesequential downsampling and upsampling of the medical imaging datacomprises: analyzing the medical imaging data based on a first filterthat comprises a first size; analyzing the medical imaging data based ona second filter that comprises a second size that is different than thefirst size; and analyzing the medical imaging data based on a thirdfilter that comprises the first size associated with the first filter.14. The method of claim 12, further comprising: performing, by thesystem, a local pooling process for an activation map associated with aconvolutional layer of the convolutional neural network prior toperforming a global pooling process associated with the convolutionalneural network.
 15. The method of claim 12, wherein the performing theiterative sequential downsampling and upsampling of the medical imagingdata comprises generating the learned medical imaging output based on afirst convolutional layer process associated with downsampling of themedical imaging data and a second convolutional layer process associatedwith upsampling of the medical imaging data.
 16. The method of claim 12,further comprising: generating, by the system, the learned medicalimaging output based on a class activation mapping process that appliesa set of weights to a set of heat maps associated with the medicalimaging data.
 17. The method of claim 12, further comprising: merging,by the system, a set of classifier layers associated with theconvolutional neural network and a set of activation maps associatedwith the convolutional neural network to generate the learned medicalimaging output.
 18. A method, comprising: receiving, by a systemcomprising a processor, medical imaging data that comprises a set ofmedical images; training, by the system, a convolutional neural networkby performing iterative sequential downsampling and upsampling of themedical imaging data associated with convolutional layers of theconvolutional neural network; and generating, by the system, a set offilter values for the convolutional neural network based on theiterative sequential downsampling and upsampling of the medical imagingdata.
 19. The method of claim 18, wherein the training comprises:analyzing the medical imaging data based on a first convolutional layerfilter that comprises a first size; analyzing the medical imagingdatabase on a second convolutional layer filter that comprises a secondsize that is different than the first size; and analyzing the medicalimaging database on a third convolutional layer filter that comprisesthe first size associated with the first convolutional layer filter. 20.The method claim 18, wherein the generating the set of filter valuescomprises generating a set of weights for the convolutional neuralnetwork based on the iterative sequential downsampling and upsampling ofthe medical imaging data.